Non-zero rounding and prediction mode selection techniques in video encoding

ABSTRACT

In one aspect of this disclosure, techniques are described for selecting among default weighted prediction, implicit weighted prediction, and explicit weighted prediction. In this context, techniques are also described for adding offset to prediction data, e.g., using the format of explicit weighted prediction to allow for offsets to predictive data that is otherwise determined by implicit or default weighted prediction.

The following co-pending and commonly assigned application is expresslyincorporated herein by reference: “Non-Zero Rounding And Prediction ModeSelection Techniques In Video Encoding” filed on the same day as thisapplication for Marta Karczewicz, Peisong Chen and Yan Ye, and bearingAttorney Docket Number 082069U1.

TECHNICAL FIELD

The disclosure relates to video encoding and, more particularly, videoencoding techniques that use bi-directional prediction.

BACKGROUND

Digital multimedia capabilities can be incorporated into a wide range ofdevices, including digital televisions, digital direct broadcastsystems, wireless communication devices, wireless broadcast systems,personal digital assistants (PDAs), laptop or desktop computers, digitalcameras, digital recording devices, video gaming devices, video gameconsoles, cellular or satellite radio telephones, digital media players,and the like. Digital multimedia devices may implement video codingtechniques, such as MPEG-2, ITU-H.263, MPEG-4, or ITU-H.264/MPEG-4 Part10, Advanced Video Coding (AVC), to transmit and receive or store andretrieve digital video data more efficiently. Video encoding techniquesmay perform video compression via spatial and temporal prediction toreduce or remove redundancy inherent in video sequences.

In video encoding, the compression often includes spatial prediction,motion estimation and motion compensation. Intra-coding relies onspatial prediction and transform coding, such as discrete cosinetransform (DCT), to reduce or remove spatial redundancy between videoblocks within a given video frame. Inter-coding relies on temporalprediction and transform coding to reduce or remove temporal redundancybetween video blocks of successive video frames of a video sequence.Intra-coded frames (“I-frames”) are often used as random access pointsas well as references for the inter-coding of other frames. I-frames,however, typically exhibit less compression than other frames. The termI-units may refer to I-frames, I-slices or other independently decodableportions of an I-frame.

For inter-coding, a video encoder performs motion estimation to trackthe movement of matching video blocks between two or more adjacentframes or other coded units, such as slices of frames. Inter-codedframes may include predictive frames (“P-frames”), which may includeblocks predicted from a previous frame, and bidirectional predictiveframes (“B-frames”), which may include blocks predicted from a previousframe and a subsequent frame of a video sequence. The terms P-frames andB-frames are somewhat historic in the sense that early coding techniqueslimited prediction in specific directions. Newer coding formats andstandards may not limit the prediction direction of P-frames orB-frames. Thus, the term “bi-directional” now refers to prediction basedon two or more lists of reference data regardless of the temporalrelationship of such reference data relative to the data being coded.

Consistent with newer video standards such as ITU H.264, for example,bi-directional prediction may be based on two different lists which donot necessarily need to have data that resides temporally before andafter the current video block. In other words, B-video blocks may bepredicted from two lists of data, which may correspond to data from twoprevious frames, two subsequent frames, or one previous frame and onesubsequent frame. In contrast, P-video blocks are predicted based on onelist, i.e., one data structure, which may correspond to one predictiveframe, e.g., one previous frame or one subsequent frame. B-frames andP-frames may be more generally referred to as P-units and B-units.P-units and B-units may also be realized in smaller coded units, such asslices of frames or portions of frames. B-units may include B-videoblocks, P-video blocks or I-video blocks. P-units may include P-videoblocks or I-video blocks. I-units may include only I-video blocks.

For P- and B-video blocks, motion estimation generates motion vectors,which indicate the displacement of the video blocks relative tocorresponding prediction video blocks in predictive reference frame(s)or other coded units. Motion compensation uses the motion vectors togenerate prediction video blocks from the predictive reference frame(s)or other coded units. After motion compensation, a residual video blockis formed by subtracting the prediction video block from the originalvideo block to be coded. The video encoder usually applies transform,quantization and entropy coding processes to further reduce the bit rateassociated with communication of the residual block. I-and P-units arecommonly used to define reference blocks for the inter-coding of P- andB-units.

SUMMARY

This disclosure describes video encoding and decoding techniquesapplicable to bi-directional prediction. In bi-directional prediction, avideo block may be predictively encoded and decoded based on twodifferent lists of predictive reference data. In one aspect of thisdisclosure, techniques are described for selecting among defaultweighted prediction, implicit weighted prediction, and explicit weightedprediction. In this context, techniques are also described for addingoffset to prediction data, e.g., using the format of explicit weightedprediction to allow for offsets to predictive data that is otherwisedefined by implicit or default weighted prediction.

In one example, this disclosure describes a method of encoding videodata, the method comprising selecting between default weightedprediction and implicit weighted prediction, upon selecting defaultweighted prediction, selecting between explicit weighted predictionbased on calculated weights and explicit weighted prediction based onweights defined by the default weighted prediction, upon selectingimplicit weighted prediction, selecting between explicit weightedprediction based on calculated weights and explicit weighted predictionbased on weights defined by the implicit weighted prediction, andencoding the video data using weighed prediction data defined by theselections.

In another example, this disclosure describes a video encoder apparatusthat encodes video data, the video encoder apparatus comprising a memorythat stores the video data, and a motion compensation unit that selectsbetween default weighted prediction and implicit weighted prediction,upon selecting default weighted prediction, selects between explicitweighted prediction based on calculated weights and explicit weightedprediction based on weights defined by the default weighted prediction,upon selecting implicit weighted prediction, selects between explicitweighted prediction based on calculated weights and explicit weightedprediction based on weights defined by the implicit weighted prediction,wherein the video encoder apparatus encodes the video data using weighedprediction data defined by the selections.

In another example, this disclosure describes a device that encodesvideo data, the device comprising means for selecting between defaultweighted prediction and implicit weighted prediction, upon selectingdefault weighted prediction, means for selecting between explicitweighted prediction based on calculated weights and explicit weightedprediction based on weights defined by the default weighted prediction,upon selecting implicit weighted prediction, means for selecting betweenexplicit weighted prediction based on calculated weights and explicitweighted prediction based on weights defined by the implicit weightedprediction, and means for encoding the video data using weighedprediction data defined by the selections.

The techniques described in this disclosure may be implemented inhardware, software, firmware, or any combination thereof If implementedin software, the software may be executed in one or more processors,such as a microprocessor, application specific integrated circuit(ASIC), field programmable gate array (FPGA), or digital signalprocessor (DSP). The software that executes the techniques may beinitially stored in a computer-readable medium and loaded and executedin the processor.

Accordingly, this disclosure also contemplates a computer-readablestorage medium comprising instructions that when executed cause aprocessor to select between default weighted prediction and implicitweighted prediction, upon selecting default weighted prediction, selectbetween explicit weighted prediction based on calculated weights andexplicit weighted prediction based on weights defined by the defaultweighted prediction, upon selecting implicit weighted prediction, selectbetween explicit weighted prediction based on calculated weights andexplicit weighted prediction based on weights defined by the implicitweighted prediction, and encode the video data using weighed predictiondata defined by the selections.

The details of one or more aspects of the disclosure are set forth inthe accompanying drawings and the description below. Other features,objects, and advantages of the techniques described in this disclosurewill be apparent from the description and drawings, and from the claims.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating one exemplary video encoding anddecoding system that may implement techniques of this disclosure.

FIG. 2 is a block diagram illustrating an example of a video encoderthat may perform offset techniques consistent with this disclosure.

FIG. 3 is a block diagram illustrating an example of a motioncompensation unit in more detail.

FIG. 4 is an example of a video decoder that may perform offsettechniques consistent with this disclosure.

FIG. 5 is a flow chart illustrating an exemplary process performed by avideo encoder consistent with this disclosure.

FIG. 6 is a flow chart illustrating an exemplary process performed by avideo decoder consistent with this disclosure.

FIG. 7 is a flow chart illustrating another exemplary process performedby a video encoder consistent with this disclosure.

DETAILED DESCRIPTION

This disclosure describes video encoding and decoding techniquesapplicable to bi-directional prediction. In bi-directional prediction, avideo block is predictively encoded and decoded based on two differentlists of predictive reference data. In one aspect of this disclosure,rounding adjustments to bi-directional predictive data may be purposelyeliminated to provide predictive data that lacks any rounding bias. Inthis case, rounded and unrounded predictive data may both be consideredin a rate-distortion analysis to identify the best data for predictionof a given video block. One or more syntax elements may be encoded toindicate the selection, and a decoder may interpret the one or moresyntax elements in order to determine whether rounding should be used inthe decoding process.

In another aspect of this disclosure, encoding techniques are describedfor selecting among default weighted prediction, implicit weightedprediction, and explicit weighted prediction. In this context,techniques are also described for adding offset to prediction data,e.g., using the format of explicit weighted prediction to allow foroffsets to predictive data that is otherwise determined by implicit ordefault weighted prediction.

Weighted prediction refers to bi-directional prediction in which weightscan be assigned to two or more different sets of predictive data.Default weighted prediction refers to weighted prediction in which theweight factors associated with two or more different lists arepre-defined by some default setting. Default weighted prediction, insome cases, may assign equal weightings to each of the lists. Implicitweighted prediction refers to weighted prediction in which the weightfactors associated with two or more different lists are defined based onsome implicit factors associated with the data. For example, implicitweight factors may be defined by the temporal positions of the data inthe two different lists relative to the data being predictively coded.If data in list 0 is temporally closer to the data being predictivelycoded than data in list 1, then the data is list 0 may be assigned agreater implicit weigh factor in implicit weighted prediction.

Explicit weighted prediction refers to weighted prediction in which theweight factors are dynamically defined as part of the coding process,and encoded into the bitstream. Explicit weighted prediction isdifferent than default weighted prediction and implicit weightedprediction in this respect, e.g., explicit weighted prediction resultsin weight factors that are encoded as part of the bitstream whereasdefault and implicit weighted prediction define weight factors that areeither pre-defined or determined by the decoder without the weightfactors being in the bitstream.

In accordance with one aspect of this disclosure, weighted predictionmay be modified relative to conventional weighted prediction byeliminating rounding adjustments to the weighted prediction data. Inthis case, the encoder may analyze and consider weighted prediction datathat is both rounded and unrounded, and may use either the rounded orunrounded weighted prediction data based on a rate-distortion analysis.One or more syntax elements may be defined and encoded into thebitstream in order to identify whether the rounded or unrounded weightedprediction data was used. The decoder may interpret the one or moresyntax elements in order to determine whether rounded or unroundedweighted prediction data should be used in the decoding.

In accordance with another aspect of this disclosure, encodingtechniques are described for selecting among default weightedprediction, implicit weighted prediction, and explicit weightedprediction. Rather than consider each of these possibilities relative toone another, an encoder may be programmed to select between defaultweighted prediction and implicit weighted prediction. The selection,then, may be compared to explicit weighted prediction. Explicit weightedprediction may be performed to calculate explicit weight factors, butthe computed explicit weighted prediction may also be compared toexplicit weighted prediction having weight factors that correspond tothose defined by either default weighted prediction or implicit weightedprediction.

Explicit weighted prediction may have an advantage relative to defaultweighted prediction and implicit weighted prediction in that explicitweighted prediction allows for an offset to be added to the predictivedata. The offset may bias or adjust the predictive data and may be veryuseful in accounting for flashes, a darkening sky, scene changes, orother types of illumination changes between video frames. The offset,for example, may provide a common adjustment for all values of a videoblock, e.g., to bias the values upward or downward. In accordance withthis disclosure, the weight factors defined by either default weightedprediction or implicit weighted prediction may be considered in thecontext of explicit weighted prediction, thereby facilitating theaddition of offset while maintaining weight factors associated withdefault or implicit weighted prediction. In this manner, predictive datamay be improved, which may help to improve data compression in somecases.

FIG. 1 is a block diagram illustrating one exemplary video encoding anddecoding system 10 that may implement techniques of this disclosure. Asshown in FIG. 1, system 10 includes a source device 12 that transmitsencoded video to a destination device 16 via a communication channel 15.Source device 12 and destination device 16 may comprise any of a widerange of devices. In some cases, source device 12 and destination device16 comprise wireless communication devices, such as wireless handsets,so-called cellular or satellite radiotelephones, or any devices that cancommunicate video information over a communication channel 15, which mayor may not be wireless. The techniques of this disclosure, however,which concern non-zero rounding and prediction mode selectiontechniques, are not necessarily limited to wireless applications orsettings.

In the example of FIG. 1, source device 12 may include a video source20, video encoder 22, a modulator/demodulator (modem) 23 and atransmitter 24. Destination device 16 may include a receiver 26, a modem27, a video decoder 28, and a display device 30. In accordance with thisdisclosure, video encoder 22 of source device 12 may be configured toapply non-zero rounding and prediction mode selection techniques as partof a video encoding process. Video decoder 28 may receive one or moresyntax elements indicating the selection and indicating whether non-zerorounding was used. Accordingly, video decoder 28 may perform the properweighted prediction signaled in the received bistream.

The illustrated system 10 of FIG. 1 is merely exemplary. The non-zerorounding and prediction mode selection techniques of this disclosure maybe performed by any coding device that supports bi-directional motioncompensated prediction. Source device 12 and destination device 16 aremerely examples of such coding devices in which source device 12generates coded video data for transmission to destination device 16. Insome cases, devices 12, 16 may operate in a substantially symmetricalmanner such that, each of devices 12, 16 includes video encoding anddecoding components. Hence, system 10 may support one-way or two-wayvideo transmission between video devices 12, 16, e.g., for videostreaming, video playback, video broadcasting, or video telephony.

Video source 20 of source device 12 may include a video capture device,such as a video camera, a video archive containing previously capturedvideo, or a video feed from a video content provider. As a furtheralternative, video source 20 may generate computer graphics-based dataas the source video, or a combination of live video, archived video, andcomputer-generated video. In some cases, if video source 20 is a videocamera, source device 12 and destination device 16 may form so-calledcamera phones or video phones. In each case, the captured, pre-capturedor computer-generated video may be encoded by video encoder 22. Theencoded video information may then be modulated by modem 23 according toa communication standard, e.g., such as code division multiple access(CDMA) or another communication standard, and transmitted to destinationdevice 16 via transmitter 24. Modem 23 may include various mixers,filters, amplifiers or other components designed for signal modulation.Transmitter 24 may include circuits designed for transmitting data,including amplifiers, filters, and one or more antennas.

Receiver 26 of destination device 16 receives information over channel15, and modem 27 demodulates the information. Again, the video encodingprocess may implement one or more of the techniques described herein toprovide non-zero rounding and prediction mode selection consistent withthis disclosure. The information communicated over channel 15 mayinclude information defined by video encoder 22, which may be used byvideo decoder 28 consistent with this disclosure. Display device 30displays the decoded video data to a user, and may comprise any of avariety of display devices such as a cathode ray tube, a liquid crystaldisplay (LCD), a plasma display, an organic light emitting diode (OLED)display, or another type of display device.

In the example of FIG. 1, communication channel 15 may comprise anywireless or wired communication medium, such as a radio frequency (RF)spectrum or one or more physical transmission lines, or any combinationof wireless and wired media. Accordingly, modem 23 and transmitter 24may support many possible wireless protocols, wired protocols or wiredand wireless protocols. Communication channel 15 may form part of apacket-based network, such as a local area network (LAN), a wide-areanetwork (WAN), or a global network, such as the Internet, comprising aninterconnection of one or more networks. Communication channel 15generally represents any suitable communication medium, or collection ofdifferent communication media, for transmitting video data from sourcedevice 12 to destination device 16. Communication channel 15 may includerouters, switches, base stations, or any other equipment that may beuseful to facilitate communication from source device 12 to destinationdevice 16.

Video encoder 22 and video decoder 28 may operate according to a videocompression standard, such as the ITU-T H.264 standard, alternativelydescribed as MPEG-4, Part 10, Advanced Video Coding (AVC). Thetechniques of this disclosure, however, are not limited to anyparticular coding standard. Although not shown in FIG. 1, in someaspects, video encoder 22 and video decoder 28 may each be integratedwith an audio encoder and decoder, and may include appropriate MUX-DEMUXunits, or other hardware and software, to handle encoding of both audioand video in a common data stream or separate data streams. Ifapplicable, MUX-DEMUX units may conform to the ITU H.223 multiplexerprotocol, or other protocols such as the user datagram protocol (UDP).

The ITU-T H.264/MPEG-4 (AVC) standard was formulated by the ITU-T VideoCoding Experts Group (VCEG) together with the ISO/IEC Moving PictureExperts Group (MPEG) as the product of a collective partnership known asthe Joint Video Team (JVT). In some aspects, the techniques described inthis disclosure may be applied to devices that generally conform to theH.264 standard. The H.264 standard is described in ITU-T RecommendationH.264, Advanced Video Coding for generic audiovisual services, by theITU-T Study Group, and dated March, 2005, which may be referred toherein as the H.264 standard or H.264 specification, or the H.264/AVCstandard or specification. The Joint Video Team (JVT) continues to workon extensions to H.264/MPEG-4 AVC.

Work to advance the H.264/MPEG-4 AVC standard takes place in variousforums of the ITU-T, such as the Key Technologies Area (KTA) forum. TheKTA forum seeks, in part, coding technologies that exhibit higher codingefficiency than that exhibited by the H.264/AVC standard. The techniquesdescribed in this disclosure may provide for coding improvementsrelative to the H.264/AVC standard. Recently, the KTA forum received adocument detailing techniques that may be considered relevant or relatedto the techniques described herein, as a submission numbered VCEG-AI33,titled “High Precision Interpolation and Prediction,” by Yan Ye, PeisongChen, and Marta Karczewicz and presented at the 35^(th) Meeting inBerlin, Germany, occurring on Jul. 16-18, 2008, which is herebyincorporated by reference in its entirety.

Video encoder 22 and video decoder 28 each may be implemented as one ormore microprocessors, digital signal processors (DSPs), applicationspecific integrated circuits (ASICs), field programmable gate arrays(FPGAs), discrete logic, software, hardware, firmware or anycombinations thereof. Each of video encoder 22 and video decoder 28 maybe included in one or more encoders or decoders, either of which may beintegrated as part of a combined encoder/decoder (CODEC) in a respectivemobile device, subscriber device, broadcast device, server, or the like.

A video sequence typically includes a series of video frames. Videoencoder 22 and video decoder 28 may operate on video blocks withinindividual video frames in order to encode and decode the video data.The video blocks may have fixed or varying sizes, and may differ in sizeaccording to a specified coding standard. Each video frame may include aseries of slices or other independently decodable units. Each slice mayinclude a series of macroblocks, which may be arranged into sub-blocks.As an example, the ITU-T H.264 standard supports intra prediction invarious block sizes, such as 16 by 16, 8 by 8, or 4 by 4 for lumacomponents, and 8×8 for chroma components, as well as inter predictionin various block sizes, such as 16 by 16, 16 by 8, 8 by 16, 8 by 8, 8 by4, 4 by 8 and 4 by 4 for luma components and corresponding scaled sizesfor chroma components. Video blocks may comprise blocks of pixel data,or blocks of transformation coefficients, e.g., following atransformation process such as discrete cosine transform or aconceptually similar transformation process.

Smaller video blocks can provide better resolution, and may be used forlocations of a video frame that include high levels of detail. Ingeneral, macroblocks and the various sub-blocks may be considered to bevideo blocks. In addition, a slice may be considered to be a series ofvideo blocks, such as macroblocks and/or sub-blocks. Each slice may bean independently decodable unit of a video frame. Alternatively, framesthemselves may be decodable units, or other portions of a frame may bedefined as decodable units. The term “coded unit” refers to anyindependently decodable unit of a video frame such as an entire frame, aslice of a frame, a group of pictures (GOPs), or another independentlydecodable unit defined according to the coding techniques used.

Following inter-based predictive encoding (which includes interpolationand the techniques of this disclosure to efficiently select a predictionalgorithm or mode by which to predict a coded unit), and following anytransforms (such as the 4×4 or 8×8 integer transform used in H.264/AVCor a discrete cosine transform or DCT), quantization may be performed.Quantization generally refers to a process in which coefficients arequantized to possibly reduce the amount of data used to represent thecoefficients. The quantization process may reduce the bit depthassociated with some or all of the coefficients. For example, a 16-bitvalue may be rounded down to a 15-bit value during quantization.Following quantization, entropy coding may be performed, e.g., accordingto content adaptive variable length coding (CAVLC), context adaptivebinary arithmetic coding (CABAC), or another entropy coding methodology.

The techniques of this disclosure are specifically applicable toweighted bi-directional prediction. As mentioned above, bi-directionalprediction is prediction of so-called “B-video blocks” based on twodifferent lists of data. B-video blocks may be predicted from two listsof data from two previous frames, two lists of data from subsequentframes, or one list of data from a previous frame and one from asubsequent frame. In contrast, P-video blocks are predicted based on onelist, which may correspond to one predictive frame, e.g., one previousframe or one subsequent frame. B-frames and P-frames may be moregenerally referred to as P-units and B-units. P-units and B-units mayalso be realized in smaller coded units, such as slices of frames orportions of frames. B-units may include B-video blocks, P-video blocksor I-video blocks. P-units may include P-video blocks or I-video blocks.I-units may include only I-video blocks.

Weighted bi-directional prediction refers to bi-directional predictionthat allows for weight factors to be assigned to the two differentlists. Each list may comprise a set of data associated with a predictiveframe or other coded unit. In weighted bi-directional prediction onelist may be weighted more heavily in generating predictive data. If oneof the lists has data that is more similar to the video block beingcoded, for example, then that list may be weighted more heavily than theother list.

For different types of weighted bi-directional prediction in accordancewith ITU-T H.264, video encoder 22 and video decoder 28 may generallysupport three different types of prediction modes. A first predictionmode, referred to as “default weighted prediction,” refers to weightedprediction in which the weight factors associated with two or moredifferent lists are pre-defined by some default setting. Defaultweighted prediction, in some cases, may assign equal weightings to eachof the lists.

A second prediction mode, referred to as “implicit weighted prediction,”refers to weighted prediction in which the weight factors associatedwith two or more different lists are defined based on some implicitfactors associated with the data. For example, implicit weight factorsmay be defined by the relative temporal positions of the data in the twodifferent lists relative to the data being predictively coded. In bothdefault weighted prediction and implicit weighted prediction, the weightfactors are not included in the bitstream. Instead, video decoder 28 maybe programmed to know the weight factors (for default) or may beprogrammed to know how to derive the weight factors (for implicit).

A third prediction mode, referred to as “explicit weighted prediction,”refers to weighted prediction in which the weight factors aredynamically defined as part of the coding process, and encoded into thebitstream. Explicit weighted prediction is different than defaultweighted prediction and implicit weighted prediction in this respect,e.g., explicit weighted prediction results in weight factors that areencoded as part of the bitstream.

In accordance with one aspect of this disclosure, weighted predictionmay be modified relative to conventional weighted prediction byeliminating rounding adjustments to the weighted prediction data. Inthis case, encoder 22 may analyze and consider weighted prediction datathat is both rounded and unrounded, and may use either the rounded orunrounded weighted prediction data based on a rate-distortion analysis.One or more syntax elements may be defined and encoded into thebitstream in order to identify whether the rounded or unrounded weightedprediction data was used. Decoder 28 may decode and interpret the syntaxelement(s), and based on the syntax element(s), decoder 28 may useeither the rounded or unrounded weighted prediction data in the decodingprocess. The removal of rounding adjustments is applicable to defaultweighted prediction, implicit weighted prediction and explicit weightedprediction.

In accordance with another aspect of this disclosure, video encoder 22may select among default weighted prediction, implicit weightedprediction, and explicit weighted prediction. In this aspect, ratherthan consider each of these possibilities relative to one another,encoder 22 may be programmed to select between default weightedprediction and implicit weighted prediction. The selection, then, may becompared to explicit weighted prediction. Specifically, encoder 22 mayperform explicit weighted prediction to calculate explicit weightfactors, but encoder 22 may also compare the computed explicit weightedprediction to explicit weighted prediction having weight factors thatcorrespond to those defined by either default weighted prediction orimplicit weighted prediction.

Explicit weighted prediction may have an advantage relative to defaultweighted prediction and implicit weighted prediction in that explicitweighted prediction allows for an offset to be encoded. The offset mayadjust the predictive data and may be very useful in accounting forflashes, a darkening sky, scene changes, or other types of illuminationchanges between video frames. In accordance with this disclosure, theweight factors defined by either default weighted prediction or implicitweighted prediction may be considered by video encoder 22 in the contextof explicit weighted prediction, thereby facilitating the addition ofoffset while maintaining weight factors associated with default orimplicit weighted prediction. In this manner, predictive data may beimproved in some cases, which may help to improve data compression.

In the context of video encoding, video encoder 22 can calculate a DCoffset by first averaging luma pixel values for a luma video block beingencoded. Video encoder 22 may next average luma pixel values for thepredictive video block used to encode the video block. Each of thesecalculated values may comprise a DC value. Video encoder 22 may computethe DC offset by subtracting the DC values from one another, e.g., bysubtracting the average luma value of the current block being coded fromthe average luma value of the predictive block used to encode thecurrent block. DC offsets might also be defined for chroma components,if desired. DC offsets may also be accumulated over a given coded unit(e.g., a frame or a slice) and defined for a coded unit as the averageof offsets for all of the blocks of a given coded unit.

FIG. 2 is a block diagram illustrating an example of a video encoder 50that may perform techniques consistent with this disclosure. Videoencoder 50 may correspond to video encoder 22 of source device 12, or avideo encoder of a different device. Video encoder 50 may perform intra-and inter-coding of blocks within video frames, although intra-codingcomponents are not shown in FIG. 2 for ease of illustration.Intra-coding relies on spatial prediction to reduce or remove spatialredundancy in video within a given video frame. Inter-coding relies ontemporal prediction to reduce or remove temporal redundancy in videowithin adjacent frames of a video sequence. Intra-mode (I-mode) mayrefer to the spatial based compression mode and Inter-modes such asprediction (P-mode) or bi-directional (B-mode) may refer to the temporalbased compression modes.

As shown in FIG. 2, video encoder 50 receives a current video blockwithin a video frame or slice to be encoded. In the example of FIG. 2,video encoder 50 includes motion estimation unit 32, a motioncompensation unit 35, a memory 34, an adder 48, a transform unit 38, aquantization unit 40, and an entropy coding unit 46. For video blockreconstruction, video encoder 50 also includes an inverse quantizationunit 42, an inverse transform unit 44, and an adder 51. Video encoder 50may also include a deblocking filter (not shown) to filter blockboundaries to remove blockiness artifacts from reconstructed video. Ifdesired, the deblocking filter would typically filter the output ofadder 51.

During the encoding process, video encoder 50 receives a video block tobe coded, and motion estimation unit 32 and motion compensation unit 35perform inter-predictive coding. Motion estimation unit 32 and motioncompensation unit 35 may be highly integrated, but are illustratedseparately for conceptual purposes. Motion estimation is typicallyconsidered the process of generating motion vectors, which estimatemotion for video blocks. A motion vector, for example, may indicate thedisplacement of a predictive block within a predictive frame (or othercoded unit) relative to the current block being coded within the currentframe (or other coded unit). Motion compensation is typically consideredthe process of fetching or generating the predictive block based on themotion vector determined by motion estimation. Again, motion estimationunit 32 and motion compensation unit 35 may be functionally integrated.For demonstrative purposes, the techniques described in this disclosureare described as being performed by motion compensation unit 35.

Motion estimation unit 32 selects the appropriate motion vector for thevideo block to be coded by comparing the video block to video blocks ofone or more predictive coded units (e.g., a previous and/or future framein terms of time or temporally). Motion estimation unit 32 may, as anexample, select a motion vector for a B-frame in a number of ways. Inone way, motion estimation unit 32 may select a previous or future framefrom a first set of frames (referred to as list 0) and determine amotion vector using only this previous or future frame from list 0.Alternatively, motion estimation unit 32 may select a previous or futureframe from a second set of frames (referred to as list 1) and determinea motion vector using only this previous or future frame from list 1. Inyet another way, motion estimation unit 32 may select a first frame fromlist 0 and a second frame from list 1 and select one or more motionvectors from the first frame of list 0 and the second frame of list 1.This third form of prediction may be referred to as bi-predictive motionestimation. Techniques of this disclosure may be implemented so as toefficiently select a motion-compensated bi-prediction mode. The selectedmotion vector for any given list may point to a predictive video blockthat is most similar to the video block being coded, e.g., as defined bya metric such as sum of absolute difference (SAD) or sum of squareddifference (SSD) of pixel values of the predictive block relative topixel values of the block being coded.

According to the ITU-T H.264/AVC standard, three motion-compensatedbi-predictive algorithms or modes may be used to predict a B-frame orportions thereof, such as video blocks, macroblocks or any otherdiscreet and/or contiguous portion of a B-frame. A firstmotion-compensated bi-predictive algorithm or mode, which is commonlyreferred to as default weighted prediction, may involve applying defaultweights to each identified video block of the first frame of list 0 andthe second frame of list 1. The default weights may be programmedaccording to the standard, and are often selected to be equal fordefault weighted prediction. The weighted blocks of the first and secondframes are then added together and divided by the total number of framesused to predict the B-frame, e.g., two in this instance. Often, thisdivision is accomplished by adding 1 to the addition of the weightedblocks of the first and second frames and then shifting the result tothe right by one bit. The addition of 1 is a rounding adjustment.

In accordance with one aspect of this disclosure, the addition of 1 (therounding adjustment) prior to the right shift by one may be avoided,thus eliminating upward biased rounding. Motion compensation unit 35 maygenerate both a weighted block with rounding and a weighted blockwithout rounding, and may select the block that achieves the best codingefficiency.

More generally, weighted prediction may be given by:

pred(i,j)=(pred0(i,j)*w0+pred1(i,j)*w1+2^(r))>>(r+1)

where pred(i,j) is data associated with the weighed prediction block,pred0(i,j) is data from list 0, pred1(i,j) is data from list 1, w0 andw1 are the weight factors, 2^(r) is the rounding term, and >> is a rightshift operation by (r+1) bits. Consistent with this disclosure, twodifferent version of pred(i,j) may be generated and considered by motioncompensation unit 35. The first is consistent with the equation above,and the second is consistent with the equation above without rounding,i.e., with the term “2^(r)” removed from the equation. Eliminating thisrounding may achieve better weighted predictive data in some cases,which can improve coding efficiency. Motion compensation unit 35 maygenerate one or more syntax elements to define whether or not roundingwas used for a given video block or set of video blocks. Both thebi-directional prediction mode and the one or more syntax elementsindicating whether rounding was used may be output form motioncompensation unit 35 to entropy coding unit 46 for inclusion in theencoded bitstream.

B pictures use two lists of previously-coded reference pictures, list 0and list 1. These two lists can each contain past and/or future codedpictures in temporal order. Blocks in a B picture may be predicted inone of several ways: motion-compensated prediction from a list 0reference picture, motion-compensated prediction from a list 1 referencepicture, or motion-compensated prediction from the combination of bothlist 0 and list 1 reference pictures. To get the combination of bothlist 0 and list 1 reference pictures, two motion compensated referenceareas are obtained from list 0 and list 1 reference picturerespectively. Their combination will be used to predict the currentblock.

In this disclosure, the term “B pictures” will be used to refergenerally to any types of B units, which may be B frames, B slices, orpossibly other video units that include at least some B video blocks. Asmentioned, B pictures may allow 3 types of weighted prediction. Forsimplicity, only forward prediction in unidirectional prediction isshown below, although backwards prediction could also be used.

Default weighted prediction may be defined by the following equationsfor unidirectional prediction and bidirectional prediction,respectively.

Unidirectional prediction: pred(i,j)=pred0(i,j)

Bidirectional prediction: pred(i,j)=(pred0(i,j)+pred1(i,j)+1)>>1

where pred0(i,j) and pred1(i,j) are prediction data from list 0 and list1.

Implicit weighted prediction may be defined by the following equationsfor unidirectional prediction and bidirectional prediction,respectively.

Unidirectional prediction: pred(i,j)=pred0(i,j)

Bidirectional prediction: pred(i,j)=(pred0(i,j)*w0+pred1(i,j)*w1+32)>>6

In this case, each prediction is scaled by a weighting factor w0 or w1,where w0 and w1 are calculated based on the relative temporal positionof the list 0 and list 1 reference pictures.

Explicit weighted prediction may be defined by the following equationsfor unidirectional prediction and bidirectional prediction,respectively.

Unidirectional prediction: pred(i,j)=(pred0(i,j)*w0+2^(r−1))>>r+o1

Bidirectional prediction:pred(i,j)=(pred0(i,j)*w0+pred1(i,j)*w1+2^(r))>>(r+1)+((o1+o2+1)>>1)

In this case, the weighting factors are determined by the encoder andtransmitted in the slice header, and o1 and o2 are picture offsets forlist 0 and list 1 reference pictures respectively.

Conventionally, rounding adjustments are always used in bidirectionalprediction. According to the equations above, a rounding adjustment of 1is used in the default weighted prediction prior to a right shift byone, and a rounding adjustment of 32 is used in the implicit weightedprediction prior to a right shift by six. Generally, a roundingadjustment of 2^(r−1) is commonly used prior to a right shift by r,where r represents a positive integer.

Such frequent and biased rounding operations can reduce precision ofprediction. Furthermore, in the bidirectional prediction of the explicitweighted prediction, there are actually 2 roundings, one for thereference pictures and the other for the offsets. Therefore, in thiscase, the rounding error can accumulate. In accordance with one aspectof this disclosure, instead of doing 2 separate roundings, the videoencoder can add the offsets to the weighted prediction before rightshift as the following:

pred(i,j)=(pred0(i,j)*w0+pred1(i,j)*w1+((o1+o2)<<r)+2^(r))>>(r+1),

where pred(i,j) is the weighted prediction data associated withrounding, pred0(i,j) is data from list 0, pred1(i,j) is data from list1, w0 and w1 are weight factors, o1 and o2 are offsets, and r and 2^(r)are rounding terms that provide the rounding in conjunction with a rightshift operation “>>” by (r+1) bits. This may provide better predictionaccuracy. In this case, a new syntax element may also be defined toallow for combination of two different offsets (o1 and o2) into oneoffset. Furthermore, in this case, a rounding value may comprise therounding adjustment described above (e.g., 2^(r) prior to a right shiftoperation by (r+1) bits) as well as another a rounding value (“r”)associated with the offset. The equation above may also be modifiedslightly to provide higher precision for the offsets. If higherprecision for the offsets are desired, the offsets may be multiplied byfactors (such as by 2) and then rounded to an integer numbers. The leftshift may also be changed to account for this added precision to theoffset.

Another problem in explicit weighted prediction is that unidirectionalprediction and bidirectional prediction may share the same weights andoffsets. In order to have more flexibility for better prediction,unidirectional prediction and bidirectional prediction may be decoupledin accordance with this disclosure. In this case, unidirectionalprediction and bidirectional prediction may define different weights andoffsets for a given type of prediction (default, implicit or explicit).New syntax elements may be defined for explicit prediction to allow forbetter prediction. An encoder can include the syntax elements in a codedbitstream to signal different rounding modes used by the encoder, sothat the same rounding modes can be used by the decoder.

It is beneficial to adaptively select rounding adjustments. One way ofdoing this is to generate two or more different sets of predictive data(and possibly encode a video block several times) based on the two ormore different sets of predictive data. One set of the predictive datamay have nonzero rounding and the other may eliminate the rounding. Instill other examples, upward rounding, downward rounding, and norounding may be considered. Motion compensation unit 35 may generatethese different types of predictive data, and may conduct arate-distortion (RD) analysis to select the best predictive data for agiven video block.

Rate-distortion (RD) analysis is fairly common in video coding, andgenerally involves the calculation of a cost metric indicative of thecoding cost. The cost metric may balance the number of bits required forthe coding (the rate) and the level of quality associated with thecoding (the distortion). A typical rate-distortion cost calculation maygenerally correspond to the format of:

J(λ)=λR+D,

where J(λ) is the cost, R is the bit rate, D is the distortion, and λ isthe Lagrange multiplier.

One way for video encoder 50 to identify the most desired predictiondata is to use motion estimation unit 32 to first find motion vectors,and then implement motion compensation unit 35 and adder 48 to calculateprediction errors with and without rounding adjustments. Motioncompensation unit 35 may then select the prediction data that yields theleast prediction error. The prediction error can be calculated by usingthe sum of absolute difference between the prediction data and thecurrent data being encoded.

In explicit weighted prediction, motion compensation unit 35 of videoencoder 50 may implement three different modes. In all three explicitweighted prediction modes, each reference picture may have one offsetused for unidirectional prediction and each pair of reference pictureshas one offset for bidirectional prediction such as:

pred(i,j)=(pred0(i,j)*w0+pred1(i,j)*w1+(o<<r)+2^(r))>>(r+1),

where pred(i,j) is the first weighted prediction data, pred0(i,j) isdata from list 0, pred1(i,j) is data from list 1, w0 and w1 are weightfactors, o is a common offset applicable the pred0(i,j) from list 0 andthe pred1(i,j) from list 1, and r and 2^(r) are rounding terms thatprovide the rounding in conjunction with a right shift operation “>>” by(r+1) bits. The first mode may use the weights defined by defaultweighted prediction. The second may use weights defined by implicitweighted prediction. The third mode allows each reference picture tohave one weight for unidirectional prediction and each pair of referencepictures involved in bidirectional prediction to have a pair of weightsfor both reference pictures. The weights defined for the third mode maybe determined adaptively, and in some cases, the explicit weightedprediction framework may be used with weights defined by default orimplicit weighted prediction in order to allow for offset in thesecontexts. Furthermore, the weights and offsets defined in this thirdmode may be different for unidirectional and bidirectional prediction.The equation above may also be modified slightly to provide higherprecision for the offset. If higher precision for the offset is desired,the offset may be multiplied by a factor (such as by 2) and then roundedto an integer number. The left shift may also be changed to account forthis added precision to the offset, e.g., in this case, the left shiftmay changed to r−1.

In order for video encoder 50 to signal to a decoder the particular modethat was used for a given video block or set of video blocks, videoencoder 50 may implement 2 single-bit syntax elements:derived_weight_flag and poc_weight_flag. In this case, the derivedweight flag may be used to select between the first two explicitweighted prediction modes mentioned above, and the third mode andpoc_weight_flag may be used to select between the first and the secondexplicit weighted prediction modes.

In order to find the best weighted prediction, video encoder 50 mayperform multi-pass encoding and select the best mode based onrate-distortion cost. One way to do this is an exhaustive search wheremotion compensation unit 35 generates every possible weighted predictiondata and selects the best one. To reduce the complexity, however, motioncompensation unit 35 may implement additional techniques of thisdisclosure, e.g., to first select between default weighted predictionand implicit weighted prediction, and then to compare the selection toexplicit weighted prediction. Motion compensation unit 35 may calculateweights and offsets associated with explicit weighted prediction, andmay also use the explicit weighted prediction framework to add offsetsto data otherwise associated with default weighted prediction orimplicit weighted prediction, whichever was selected. Thus, there may betwo sets of offsets calculated by motion compensation unit 35. The firstset of offsets may be calculated by using known weights used in thedefault weighted prediction or the implicit weighted prediction, and thesecond set of offsets may be calculated jointly with weights as part ofa normal calculation of explicit weighted prediction, e.g., byminimizing the motion compensated prediction error.

To further reduce the complexity, during explicit weighted prediction,if the offsets are 0, motion compensation unit 35 may skip the explicitweighted prediction that uses default weights or implicit weights. Also,if the offsets are 0 and calculated weights have no change, motioncompensation unit 35 may skip the typical explicit weighted predictionthat uses calculated weights and offsets.

Once the desired prediction data is identified by motion compensationunit 35, as described herein, video encoder 50 forms a residual videoblock by subtracting the prediction data from the original video blockbeing coded. Adder 48 represents the component or components thatperform this subtraction operation. Transform unit 38 applies atransform, such as a discrete cosine transform (DCT) or a conceptuallysimilar transform, to the residual block, producing a video blockcomprising residual transform block coefficients. Transform unit 38, forexample, may perform other transforms, such as those defined by theH.264 standard, which are conceptually similar to DCT. Wavelettransforms, integer transforms, sub-band transforms or other types oftransforms could also be used. In any case, transform unit 38 appliesthe transform to the residual block, producing a block of residualtransform coefficients. The transform may convert the residualinformation from a pixel domain to a frequency domain.

Quantization unit 40 quantizes the residual transform coefficients tofurther reduce bit rate. The quantization process may reduce the bitdepth associated with some or all of the coefficients. For example, a9-bit value may be rounded down to an 8-bit value during quantization.In addition, quantization unit 40 may also quantize the differentoffsets for cases where offset is used.

Following quantization, entropy coding unit 46 entropy codes thequantized transform coefficients. For example, entropy coding unit 46may perform content adaptive variable length coding (CAVLC), contextadaptive binary arithmetic coding (CABAC), or another entropy codingmethodology. Following the entropy coding by entropy coding unit 46, theencoded video may be transmitted to another device or archived for latertransmission or retrieval. The coded bitstream may include entropy codedresidual blocks, motion vectors for such blocks, and other syntax suchas the syntax described herein.

Inverse quantization unit 42 and inverse transform unit 44 apply inversequantization and inverse transformation, respectively, to reconstructthe residual block in the pixel domain, e.g., for later use as thereference block in the manner described above. Adder 51 adds thereconstructed residual block to the motion compensated prediction blockproduced by motion compensation unit 35 to produce a reconstructed videoblock for storage in memory 34. The reconstructed video block may beused by motion estimation unit 32 and motion compensation unit 35 as areference block to inter-encode a block in a subsequent video frame.

FIG. 3 is a block diagram illustrating an example of motion compensationunit 35 of FIG. 2 in more detail. As shown in the example of FIG. 3,motion compensation unit 35 couples to memory 34, which stores first andsecond sets of coded units or reference frames as list 0 52A and list 152B. In addition, memory 34 may store the current video data 53 coded.Memory 34 may comprise a shared memory structure, or possibly severaldifferent memories, storage units, buffers, or other types of storagethat facilitates storage of any data discussed herein. List 0 52A andlist 1 52B are data associated with two different predictive units,e.g., data from two different frames or slices or macroblocks, inaccordance with bidirectional prediction. Again, bi-directionalprediction is not necessarily limited to any prediction directions, andthus list 0 52A and list 1 52B may store data from two previous framesor slices, two subsequent frames or slices, or one previous frame orslice and one subsequent frame or slice. Furthermore, in some cases,list 0 52A and/or list 1 52B could each include data associated withmultiple frames, slices or macrblocks. List 0 52A and/or list 1 52B aresimply two different sets of possible predictive data, and each list mayinclude one frame or slice, or several frames, slices or macroblocks inany direction relative to the current video block being encoded.

As shown in FIG. 3, motion compensation unit 35 includes a defaultweighted prediction unit 54, an implicit weighted prediction unit 56,and an explicit weighted prediction unit 58. Units 54, 56 and 58respectively perform default weighted prediction, implicit weightedprediction, and explicit weighted prediction as described herein.Rate-distortion (R-D) analysis unit 64 may select weighted predictiondata among these possibilities, and may implement techniques of thisdisclosure to facilitate the selection process.

Motion compensation unit 35 also includes a rounding unit 55 that causesone or more of units 54, 56 and 58 to generate both rounded andunrounded versions of the respective weighted prediction data. Again, byeliminating rounding, weighted prediction data may be improved in somecontexts.

In addition, motion compensation unit 35 includes an offset calculationunit 62, which calculates offset. According to ITU-T H.264/MPEG-4 AVCcoding format, offset is only allowed in explicit weighted prediction.Therefore, in order to consider offset in the context of defaultweighted prediction or implicit weighted prediction, the weightsdetermined by default weighted prediction unit 54 or implicit weightedprediction unit 56 may be forwarded to explicit weighted prediction unit58 along with offset determined by offset calculation unit 62. In thisway, explicit weighted prediction unit 58 can exploit the ITU-TH.264/MPEG-4 AVC coding format by adding offset to default weightedprediction data or implicit weighted prediction data for considerationby R-D analysis unit 64. In this case, explicit weighted prediction unit58 generates not only the normal explicit weighted prediction data, butalso prediction data that uses the weights determined by defaultweighted prediction unit 54 or implicit weighted prediction unit 56 inconjunction with offset determined by offset calculation unit 62.

Offset calculation unit may calculate offset as a difference between anaverage of video block values of a block being coded and an average ofvideo block values of the prediction block. Offset may be calculated forluma video blocks, and in some cases, offset may be calculated for lumavideo blocks and for chroma video blocks.

R-D analysis unit 64 may analyze the different weighted predictive data,and may select the weighted predictive data that generates the bestresults in terms of quality, or in terms of rate and distortion. R-Danalysis unit 64 outputs the selected weighted predictive data, whichmay be subtracted from the video block being coded via adder 48 (FIG.2). Syntax elements may be used to inform a decoder of the manner ormethod that should be used to generate the weighted predictive data. Thesyntax elements, for example, may indicate whether or not rounding wasused, and may indicate whether default, implicit or explicit weightedprediction should be used. If explicit weighted prediction should beused, the syntax elements may further identify the weight factors andthe offset, which again may be weight factors and offset associated withexplicit weighted prediction, or may be weight factors that wereactually defined by default weighted prediction unit 54 or implicitweighted prediction unit 56 with the addition of offset from offsetcalculation unit 62.

FIG. 4 is a block diagram illustrating an exemplary video decoder 70,which may perform the reciprocal decoding techniques to the encodingtechniques described above. Video decoder 70 may include an entropydecoding unit 72, a prediction unit 75, an inverse quantization unit 76,an inverse transform unit 78, a memory 74 and adder 79. Prediction unit75 may include a motion compensation (MC) unit 88, as well as spatialprediction components, which are not shown for simplicity and ease ofillustration.

Video decoder 70 may receive encoded video data, and one or more syntaxelements that indicate whether a rounding adjustment was used to encodethe encoded video data. MC unit 86 of prediction unit 75 may generateweighted prediction data that depends on two or more lists of data, asdescribed herein. In accordance with this disclosure, the weightedprediction data does not include the rounding adjustment if the one ormore syntax elements indicate that the rounding adjustment was not usedto encode the encoded video data. Video decoder 70 can decode the videodata using the weighted prediction data, e.g., by invoking adder 79 toadd the weighted prediction data (e.g., a prediction block) to residualdata (e.g., a residual block).

In general, entropy decoding unit 72 receives an encoded bitsteam andentropy decodes the bitstream to generate quantized coefficients, motioninformation and other syntax. The motion information (e.g., motionvectors) and other syntax are forwarded to prediction unit 75 for use ingenerating the predictive data. Prediction unit 75 performsbidirectional prediction consistent with this disclosure, avoidingrounding adjustments in some cases, and possibly implementing default,implicit or explicit weighted prediction according to the receivedsyntax elements. The syntax elements may identify the type of weightedprediction that to be used, coefficients and offset if explicit weightedprediction to be used, and whether rounding adjustments should be usedin the decoding.

The quantized coefficients are sent from entropy decoding unit 72 toinverse quantization unit 76, which performs inverse quantization.Inverse transform unit 78 then inverse transforms the de-quantizedcoefficients back to the pixel domain to generate a residual block.Adder 79 combines the prediction data (e.g., a prediction block)generated by prediction unit 75 with the residual block from inversetransform unit 78 to create a reconstructed video block, which may bestored in memory 74 and/or output from video decoder 70 as decoded videooutput.

FIG. 5 is a flow chart illustrating an exemplary process performed by avideo encoder consistent with this disclosure. FIG. 5 will be describedfrom the perspective of video encoder 50 of FIG. 2. As shown in FIG. 5,motion compensation unit 35 generates first weighted prediction datathat includes rounding (101), and generates second weighted predictiondata that does not include rounding (102). Motion compensation unit 35then selects prediction data from the first and second weightedprediction data based on a rate-distortion analysis (103). Inparticular, motion compensation unit 35 may determine cost metrics forthe first and second weighted prediction data that quantify and balancethe encoding rate and the encoding quality associated with the first andsecond weighted prediction data, and may select the prediction data withthe lowest cost. Video encoder 50 can then encode video data based onthe selected prediction data (104). For example, video encoder 50 mayinvoke adder 48 to subtract the selected prediction data from the videodata being coded, and then invoke transform unit 38 for transform,quantization unit 40 for quantization and entropy coding unit 46 forentropy coding of quantized and transformed residual coefficients. Inthis case, motion compensation unit 35 may generate one or more syntaxelements to indicate whether or not rounding was used for the predictiondata, and may forward such syntax elements to entropy coding unit 46 forinclusion in the coded bistream.

FIG. 6 is a flow chart illustrating an exemplary process performed by avideo decoder consistent with this disclosure. FIG. 6 will be describedfrom the perspective of video decoder 70 of FIG. 4. As shown in FIG. 6,video decoder receives encoded video data (111), and receives one ormore syntax elements that indicate whether a rounding adjustment wasused to encode the video data (1 12). In particular, entropy decodingunit 72 may receive an encoded bitstream that includes the video dataand the one or more syntax elements. Following entropy decoding, entropydecoding unit 72 may output the video data as quantized transformcoefficients, which are inverse quantized by unit 76 and inversetransformed by unit 78. Entropy decoding unit 72 may output syntaxelements to prediction unit, which includes the one or more syntaxelements that indicate whether a rounding adjustment was used to encodethe video data, motion vectors and possibly other syntax.

Prediction unit 75 invokes motion compensation unit 86 for block basedpredictive decoding. In doing so, motion compensation unit 86 generatesweighted prediction data based on the syntax (113). Accordingly, if theone or more syntax elements indicate that a rounding adjustment wasused, then motion compensation unit 86 generates weighted predictiondata that includes the rounding adjustment. However, if the one or moresyntax elements indicate that a rounding adjustment was not used, thenmotion compensation unit 86 generates weighted prediction data thatlacks the rounding adjustment. Video decoder 70 can then decode thevideo data using the weighted prediction data (114). In particular,video decoder 70 may invoke adder 79 to combine weighted prediction data(e.g., a prediction block) with residual video data (e.g., a residualblock) in order to generate a reconstruction of the video data (e.g., areconstructed video block).

FIG. 7 is a flow chart illustrating another exemplary process performedby a video encoder consistent with this disclosure. FIG. 7 will bedescribed from the perspective of motion compensation unit 35 of FIG. 3,which may form part of video encoder 50 of FIG. 2. As shown in FIG. 7,default weighted prediction unit 54 performs default weighted predictionwith rounding (201) and performs default weighted prediction withoutrounding (202). Rounding unit 55 may be invoked to define such roundingor lack thereof. Implicit weighted prediction unit 56 then performsimplicit weighted prediction with rounding (203) and performs implicitweighted prediction without rounding (204). Again, rounding unit 55 maybe invoked to define such rounding or lack thereof.

As explained above, default weighted prediction refers to weightedprediction in which the weight factors associated with two or moredifferent lists are pre-defined by some default setting. Defaultweighted prediction, in some cases, may assign equal weightings to eachof the lists. Implicit weighted prediction refers to weighted predictionin which the weight factors associated with two or more different listsare defined based on some implicit factors associated with the data. Forexample, implicit weight factors may be defined by the relative temporalpositions of the data in the two different lists relative to the databeing predictively coded.

R-D analysis unit 64 selects either the default weighted prediction orthe implicit weighted prediction (205). In particular, R-D analysis unit64 may select either the default weighted prediction or the implicitweighted prediction based on the quality and coding rate associated withthe different version of the prediction data. For example, R-D analysisunit 64 may consider the similarity of the different version of theprediction data relative to the video block being coded, and may selectthe version that is closest.

If R-D analysis unit 64 selects the default weighted prediction(“DEFAULT” 205), explicit weighted prediction unit 58 may be invoked tocalculate explicit weighted prediction data and compare such data toexplicit weighted prediction data that uses the default weights. Asnoted, this allows for offset in the context of default weights.Accordingly, the explicit weighted prediction can be used as a mechanismto provide different offsets to data that would otherwise be defined bydefault or implicit weighted prediction. As outlined above, explicitweighted prediction refers to weighted prediction in which the weightfactors are dynamically defined as part of the coding process, andencoded into the bitstream. Explicit weighted prediction is differentthan default weighted prediction and implicit weighted prediction inthis respect, e.g., explicit weighted prediction results in weightfactors that are encoded as part of the bitstream whereas default andimplicit weighted prediction define weight factors that are eitherpre-defined or determined by the decoder without the weight factorsbeing in the bitstream.

In particular, explicit weighted prediction unit 58 may calculateexplicit weights and explicit offset (206) using conventional explicitweighted prediction defined by ITU-T H.264. In order to calculate theexplicit weights, for example, explicit weighted prediction unit 58 mayapply a least mean squared (LMS) algorithm in order to solve theexplicit weighted prediction equations listed above for both weights andoffsets. In addition, explicit weighted prediction unit 58 may calculateoffset associated with the default weights (207). Offset calculationunit 62 may be invoked by explicit weighted prediction unit 58 in orderto calculate the offset. In particular, offset calculation unit 62 maycalculate a given offset as the average difference between pixel valuesof the video data being coded and the given version versions of weightedprediction data.

Explicit weighted prediction unit 58 can then generate two differentversions of prediction data. In particular, explicit weighted predictionunit 58 may perform explicit weighted prediction using default weightsand the corresponding offset (208), and may also perform explicitweighted prediction using calculated weights and the correspondingexplicit offset (209). These two different versions of explicit weightedprediction data (one being calculated according to normal explicitweighted prediction and the other being calculated using default weightswith the addition of offset) and the default weighted prediction with orwithout rounding can then be sent to R-D analysis unit 64. R-D analysisunit 64 may select the prediction mode based on an R-D analysis (210).In particular, R-D analysis unit 64 may select between these twodifferent versions of explicit weighted prediction data (one beingcalculated according to normal explicit weighted prediction and theother being calculated using default weights with the addition ofoffset). R-D analysis unit 64 may consider the similarity of thedifferent version of the prediction data relative to the video blockbeing coded, and may select the version that is closest.

A similar process occurs with respect to implicit weighted predictiondata when the implicit weighted prediction data is selected relative tothe default weighted prediction data. That is, if R-D analysis unit 64selects the implicit weighted prediction (“IMPLICIT” 205), explicitweighted prediction unit 58 may be invoked to calculate explicitweighted prediction data and compare such data to explicit weightedprediction data that uses the implicit weights. This allows for offsetin the context of implicit weights. In particular, explicit weightedprediction unit 58 may calculate explicit weights and explicit offset(211) using conventional explicit weighted prediction defined by ITU-TH.264. In addition, explicit weighted prediction unit 58 may calculateoffset associated with the implicit weights (212). Offset calculationunit 62 may be invoked by explicit weighted prediction unit 58 in orderto calculate the offsets as described herein.

Explicit weighted prediction unit 58 can then generate two differentversions of prediction data. In this case, explicit weighted predictionunit 58 may perform explicit weighted prediction using implicit weightsand the corresponding offset (213), and may also perform explicitweighted prediction using calculated weights and the correspondingexplicit offset (214). These two different versions of explicit weightedprediction data (one being calculated according to normal explicitweighted prediction and the other being calculated using implicitweights with the addition of offset) and the implicit weightedprediction with or without rounding can then be sent to R-D analysisunit 64. R-D analysis unit 64 may select the prediction mode based on anR-D analysis. In particular, R-D analysis unit 64 may select betweenthese two different versions of explicit weighted prediction data (onebeing calculated according to normal explicit weighted prediction andthe other being calculated using implicit weights with the addition ofoffset). R-D analysis unit 64 may consider the similarity of thedifferent version of the prediction data relative to the video blockbeing coded, and may select the version that is closest.

The techniques of this disclosure may be implemented in a wide varietyof devices or apparatuses, including a wireless handset, and integratedcircuit (IC) or a set of ICs (i.e., a chip set). Any components, modulesor units have been described provided to emphasize functional aspectsand does not necessarily require realization by different hardwareunits. The techniques described herein may also be implemented inhardware, software, firmware, or any combination thereof. Any featuresdescribed as modules, units or components may be implemented together inan integrated logic device or separately as discrete but interoperablelogic devices. In some cases, various features may be implemented as anintegrated circuit device, such as an integrated circuit chip orchipset.

If implemented in software, the techniques may be realized at least inpart by a computer-readable medium comprising instructions that, whenexecuted in a processor, performs one or more of the methods describedabove. The computer-readable medium may comprise a computer-readablestorage medium and may form part of a computer program product, whichmay include packaging materials. The computer-readable storage mediummay comprise random access memory (RAM) such as synchronous dynamicrandom access memory (SDRAM), read-only memory (ROM), non-volatilerandom access memory (NVRAM), electrically erasable programmableread-only memory (EEPROM), FLASH memory, magnetic or optical datastorage media, and the like. The techniques additionally, oralternatively, may be realized at least in part by a computer-readablecommunication medium that carries or communicates code in the form ofinstructions or data structures and that can be accessed, read, and/orexecuted by a computer.

The code or instructions may be executed by one or more processors, suchas one or more digital signal processors (DSPs), general purposemicroprocessors, an application specific integrated circuits (ASICs),field programmable logic arrays (FPGAs), or other equivalent integratedor discrete logic circuitry. Accordingly, the term “processor,” as usedherein may refer to any of the foregoing structure or any otherstructure suitable for implementation of the techniques describedherein. In addition, in some aspects, the functionality described hereinmay be provided within dedicated software modules or hardware modulesconfigured for encoding and decoding, or incorporated in a combinedvideo codec. Also, the techniques could be fully implemented in one ormore circuits or logic elements.

The disclosure also contemplates any of a variety of integrated circuitdevices that include circuitry to implement one or more of thetechniques described in this disclosure. Such circuitry may be providedin a single integrated circuit chip or in multiple, interoperableintegrated circuit chips in a so-called chipset. Such integrated circuitdevices may be used in a variety of applications, some of which mayinclude use in wireless communication devices, such as mobile telephonehandsets.

Various embodiments of the invention have been described. These andother embodiments are within the scope of the following claims.

1. A method of encoding video data, the method comprising: selectingbetween default weighted prediction and implicit weighted prediction;upon selecting default weighted prediction, selecting between explicitweighted prediction based on calculated weights and explicit weightedprediction based on weights defined by the default weighted prediction;upon selecting implicit weighted prediction, selecting between explicitweighted prediction based on calculated weights and explicit weightedprediction based on weights defined by the implicit weighted prediction;and encoding the video data using weighed prediction data defined by theselections.
 2. The method of claim 1, further comprising: encoding oneor more syntax elements to indicate whether rounding was used.
 3. Themethod of claim 1, wherein: prediction data associated with the explicitweighted prediction based on calculated weights includes a first offsetvalue, prediction data associated with the explicit weighted predictionbased on weights defined by the default weighted prediction includes asecond offset value, and prediction data associated with the explicitweighted prediction based on weights defined by the implicit weightedprediction includes a third offset value.
 4. The method of claim 1,wherein upon selecting the default weighted prediction: first predictiondata associated with the explicit weighted prediction based oncalculated weights includes a first offset value, and second predictiondata associated with the explicit weighted prediction based on weightsdefined by the default weighted prediction includes a second offsetvalue.
 5. The method of claim 1, wherein upon selecting the implicitweighted prediction: first prediction data associated with the explicitweighted prediction based on calculated weights includes a first offsetvalue, and second prediction data associated with the explicit weightedprediction based on weights defined by the implicit weighted predictionincludes a second offset value.
 6. A video encoder apparatus thatencodes video data, the video encoder apparatus comprising: a memorythat stores the video data; and a motion compensation unit that: selectsbetween default weighted prediction and implicit weighted prediction;upon selecting default weighted prediction, selects between explicitweighted prediction based on calculated weights and explicit weightedprediction based on weights defined by the default weighted prediction;upon selecting implicit weighted prediction, selects between explicitweighted prediction based on calculated weights and explicit weightedprediction based on weights defined by the implicit weighted prediction,wherein the video encoder apparatus encodes the video data using weighedprediction data defined by the selections.
 7. The video encoderapparatus of claim 6, wherein the video encoder apparatus encodes one ormore syntax elements to indicate whether rounding was used.
 8. The videoencoder apparatus of claim 6, wherein: prediction data associated withthe explicit weighted prediction based on calculated weights includes afirst offset value, prediction data associated with the explicitweighted prediction based on weights defined by the default weightedprediction includes a second offset value, and prediction dataassociated with the explicit weighted prediction based on weightsdefined by the implicit weighted prediction includes a third offsetvalue.
 9. The video encoder apparatus of claim 6, wherein upon themotion compensation unit selecting the default weighted prediction:first prediction data associated with the explicit weighted predictionbased on calculated weights includes a first offset value, and secondprediction data associated with the explicit weighted prediction basedon weights defined by the default weighted prediction includes a secondoffset value.
 10. The video encoder apparatus of claim 6, wherein uponmotion compensation unit selecting the implicit weighted prediction:first prediction data associated with the explicit weighted predictionbased on calculated weights includes a first offset value, and secondprediction data associated with the explicit weighted prediction basedon weights defined by the implicit weighted prediction includes a secondoffset value.
 11. The video encoder apparatus of claim 6, wherein thevideo encoder apparatus comprises an integrated circuit.
 12. The videoencoder apparatus of claim 6, wherein the video encoder apparatuscomprises a microprocessor.
 13. The video encoder apparatus of claim 6,wherein the video encoder apparatus comprises a wireless communicationdevice that includes a video encoder.
 14. A device that encodes videodata, the device comprising: means for selecting between defaultweighted prediction and implicit weighted prediction; upon selectingdefault weighted prediction, means for selecting between explicitweighted prediction based on calculated weights and explicit weightedprediction based on weights defined by the default weighted prediction;upon selecting implicit weighted prediction, means for selecting betweenexplicit weighted prediction based on calculated weights and explicitweighted prediction based on weights defined by the implicit weightedprediction; and means for encoding the video data using weighedprediction data defined by the selections.
 15. The device of claim 14,further comprising: means for encoding one or more syntax elements toindicate whether rounding was used.
 16. The device of claim 14, wherein:prediction data associated with the explicit weighted prediction basedon calculated weights includes a first offset value, prediction dataassociated with the explicit weighted prediction based on weightsdefined by the default weighted prediction includes a second offsetvalue, and prediction data associated with the explicit weightedprediction based on weights defined by the implicit weighted predictionincludes a third offset value.
 17. The device of claim 14, wherein uponselecting the default weighted prediction: first prediction dataassociated with the explicit weighted prediction based on calculatedweights includes a first offset value, and second prediction dataassociated with the explicit weighted prediction based on weightsdefined by the default weighted prediction includes a second offsetvalue.
 18. The device of claim 14, wherein upon selecting the implicitweighted prediction: first prediction data associated with the explicitweighted prediction based on calculated weights includes a first offsetvalue, and second prediction data associated with the explicit weightedprediction based on weights defined by the implicit weighted predictionincludes a second offset value.
 19. A computer-readable storage mediumcomprising instructions that when executed cause a processor to: selectbetween default weighted prediction and implicit weighted prediction;upon selecting default weighted prediction, select between explicitweighted prediction based on calculated weights and explicit weightedprediction based on weights defined by the default weighted prediction;upon selecting implicit weighted prediction, select between explicitweighted prediction based on calculated weights and explicit weightedprediction based on weights defined by the implicit weighted prediction;and encode the video data using weighed prediction data defined by theselections.
 20. The computer-readable medium of claim 19, furthercomprising instructions that when executed cause the processor to:encode one or more syntax elements to indicate whether rounding wasused.
 21. The computer-readable medium of claim 19, wherein: predictiondata associated with the explicit weighted prediction based oncalculated weights includes a first offset value, prediction dataassociated with the explicit weighted prediction based on weightsdefined by the default weighted prediction includes a second offsetvalue, and prediction data associated with the explicit weightedprediction based on weights defined by the implicit weighted predictionincludes a third offset value.
 22. The computer-readable medium of claim19, wherein upon selecting the default weighted prediction: firstprediction data associated with the explicit weighted prediction basedon calculated weights includes a first offset value, and secondprediction data associated with the explicit weighted prediction basedon weights defined by the default weighted prediction includes a secondoffset value.
 23. The computer-readable medium of claim 19, wherein uponselecting the implicit weighted prediction: first prediction dataassociated with the explicit weighted prediction based on calculatedweights includes a first offset value, and second prediction dataassociated with the explicit weighted prediction based on weightsdefined by the implicit weighted prediction includes a second offsetvalue.